.
IT Audit in IAAD – some suggestions Most IT audits in the IAAD revolve around data analysis using IDEA (or whatever software) and tend to end with it. Thus, rather then following the process of first carrying out a compliance test and then proceeding with substantive tests to confirm the results of compliance testing, out IT audits end with substantive procedures. procedures. There is no effort to identify identif y and evaluate the effectiveness of controls. There is no compliance audit at all. IT audit is a system s ystem based audit and must result in an assurance on the system. Data analysis throws up many data inconsistencies i nconsistencies and/or abnormalities. abnormalities. These are put as audit findings without with out any investigations. These need to be investigated. It could be a wrong audit observation (many cases) or an input mistake or a case of data overwriting without authority (a security lapse) or wrong programme logic. If it is a wrong logic, then there must be an identifiable pattern of these observations. That is backward linkage to a finding of data analysis. Impact of these observations must also be evaluated. The IT security needs to be looked into with more seriousness. Apart from access controls, we must look for logs, authorizations, trails, back-end security etc. A good and secure system must have means to identify each date entry/ modification to the individual who made it. In most cases, guidelines are prepared on bookish pattern of an IT audit. As these guidelines bear no relation to the audittee being audited, the final result lacks quality. Guidelines must be based on a careful study of the system being audited. We must also bear in mind that the t he term, “IT Audit” is a misnomer. We do not audit technology at all. Information Systems Audit (IS Audit) is a better term. The system includes manual procedures and controls as well and as such these should also be addressed through the guidelines. Mapping of business rules, change management, the re-engineering aspect etc. are other areas which can be looked l ooked into. The present paper is an effort to put through t hrough a few points which will help in preparing guidelines for an IT audit. These are, by no means, exhaustive. In some cases, none of them may apply. These points are not supposed to take away the innovative approaches, approaches, which could be special to an auditor. Hopefully over a period of time, more points will be added. An IT Auditor must give an assurance on the reliability of th e system. If the controls are weak or non-existent and our audit results show that the data is unreliable or is not safe and secure, this must be brought out clearly in the report. In such a scenario, where the system is not reliable, we must not make any efforts t o conclude any money value loss etc. based on analysis of such unreliable data. Such audit efforts can be reported as either the VFM aspect (if the investment is large l arge or/and the time taken is also large) or in terms of the adverse impact which such unreliable system will have on the organization. Questionnaire Page 1 of 10
.
The System and its usefulness
Before a system is developed, attempt is to be made to get the user requirements. If these requirements are not captured properly, the resultant system can never provide enough user satisfaction. Also, a documented system is easy to understand and follow. Audit should aim at assessing user’s involvement involvement in system development, availability and usability of the system s ystem related documentation and the extent of user satisfaction. 1.
Documentation
a.
Does a proper and effective documentation exist? The System Design Document (SDD), User Requirement Survey (URS) and System requirement Survey (SRS) and the User Manual documents Were these prepared (or got prepared) Are these available? Are there items (functions/processes), (functions/processes), which had been provided for in these documents (SDD, SRS and the URS), yet have not been implemented. Prepare a list of all documentation available, studying the extent of reliance of the auditee on the available documentation Does the system documentation include a data flow diagram or/and flow
b. c. d. e. f. chart? g. h.
If yes, does the flow chart /data flow diagram represent the system correctly? Are there unfulfilled user requirements? What is the extent of customer satisfaction?
2.
About completeness of the system
a.
List all outputs (reports, etc.) which the system is capable of producing. This can be compared with a list l ist of outputs which are actually being produced. Does the system produce all its output without any manual intervention? How many of these outputs are actually being used? Are these reliable, or the audittee depends on manually prepared reports. If there are reports that have never been printed even once, may examine further? Are there any reports/output, which are needed but the system is unable to produce them? If there are reports, which are not being produced, Audit must distinguish between whether the system is not capable of producing the desired report, or the system is capable, but the report is not being produced. In determining the capability of the system in producing such reports, it should be seen that the data required for producing such report is being captured/ available in the system. How many reports are still being prepared manually? If so, is it when the system has the capability to produce the reports? If yes, then why are the reports still being generated manually? Is the parallel system still functioning? If so, compare the manually prepared reports and computer generated reports.
b. c. d. e.
f.
g.
3.
The Hardware
Page 2 of 10
.
Auditing the Hardware would include the normal expenditure audit and can be extended to proper inventory control as well as upkeep of the h/w. Improper maintenance of h/w can result in lack of availability of data/information, which is one of the seven attributes of data. a.
f. g. h.
Look for inventory of assets and examine at the usage, availability, costs, maintenance practices, AMC costs etc Any approximation to the entire cost of Hardware! How have the h/w specifications been determined? Is a system of formal reporting of hardware malfunctioning in place? How many instances (say, in last 12 months) of hardware malfunctioning resulting in temporary disruption to process? What is the frequency of systems slowing down due to hardware mal functioning? How is the reason for the malfunction attributed to h/w malfunction? Are such malfunctions documented and reported to the vendor in time? Indicate the average (also the longest) duration that the system was down? Any comments on AMC
4.
The Software
b. c. d. e.
Auditing the software may involve the following a.
c. d. e.
How was it got developed (in-house, outsourced, mixed)? Any comments vis a vis these choices? Is there a prescribed procedure to document system related deficiencies for future updates? Are logs kept of any software malfunctions? Are such details available? Cost of development What about maintenance of software?
5.
The Human ware
a.
Has any work study and time and motion study been carried out determining the time required for particular quantity of work relating rel ating to data entry? Has the implementation of system resulted in saving of manpower? Has there been proper and effective training? May examine if training is being organized separately for the system managers, the Data Entr y Operators, and for the IS Security team On an average how frequently each staff member of the office is provided training Is there a procedure to document difficulties being faced by individuals while working on the system so that these can be addressed later?
b.
b. c.
d. e.
6.
Data - reliability
Data must be reliable, authentic, correct and complete. It must have proper authorization. IT auditor must observe the process of data input. Input starts with the preparation of an input document, which may take place anywhere.
Page 3 of 10
.
E.g., in the process of preparing computerized electoral roles the input document is the enumeration sheet prepared at the t he home of an individual. Observing and understanding the entire process may also reveal control weakness, weakness, if any. Supervision as a control to ensure data correctness is an ineffective control in most cases. However, we never had any comment on such issues. An input control is to be differentiated from a Validation check. A validation check, at best, can assist an input control; by itself, it cannot guarantee correctness of input. Validation is part of system design. It has to be conceived and implemented in the system. Input control can be a part of the computerized system or it can be a manual procedure etc. It will be a good idea to list all inputs that go in the system and examine each of these from compliance audit view point. See, whether adequate input controls have been provided for and are being enforced properly. The issue is if correctness of input cannot be guaranteed, the entire system is useless. A.
The Master data
The master data is of permanent nature as against a transaction data. It should not require frequent updating. In case, the master data is on more then one server, it must be consistent. 1. 2. 3. 4.
How frequently the master data is examined for its accuracy? Is the master data accurate and authentic? How often the master data is modified? While modifying the master data, is a log kept of the reasons for change, who authorized the changes, who made the changes etc.? 5. Are all changes documented with dates etc.? 6. On modification, is the original data still available in any archive or is it deleted? 7. Are there blank fields in the Master data base as it exists today? 8. In case there are more then one server, does the same master data exist on all servers? 9. Any comments on input of master data! Was it one time t ime input? 10. In cases, where a system has been up dated, the legacy data creates problems. See, how the data was transported for comments, if any and all the input controls for the new system has been applied on the data and the data is cleaned of the unwanted/irrelevant data. B.
The Input Process
Start with listing all input sources for the system. For each input, try getting the following information
Page 4 of 10
.
1. Who prepares the input document? Can the t he accuracy of this input document be certified/ assured? 2. Is there duplicity of input? 3. Is the accuracy certified before the input is made? 4. Is documentation available to verify at a later date that the input document was prepared accurately? Are there instances when an input i nput document has undergone changes changes after it i t has been input in the system? 5. Is there a prescribed procedure to modify input, if required? 6. Are there some relevant and available input documents/ sources or information which are not being captured? 7. Are the input documents in tune with the system (or vice versa) that has been developed from the point of view of capturing all details of events/transactions events/transactions and also from the point of view of ease of use? C.
Validation Checks
Properly designed validation checks (VC) assists input controls in ensuring accuracy of data. IT auditor may make an attempt to list fields available in the input document and indicate whether a validation check was provided for in the SDD and/or it exists in the t he system for this field or not. The effectiveness of a validation check may have to be assessed. D.
Input Controls
1. Start with listing Input Controls as provided for in the SDD 2. Identify and list input controls, which exist in the system 3. Check if supervision as an input control is being carried out and is effective. 4. Is there any prescribed percentage of input documents to be supervised by a senior officer? 5. Is there a procedure to document and ensure at a later stage that such supervisory checks were carried out? 6. Is there a prescribed procedure to ensure that error correction procedures based on the result of supervisory checks are carried out? 7. Are the results of supervision and the resultant corrections documented so that these can be verified at a later date? 8. Is there an attempt to carry out an analysis of input mistakes available, which can act as a management tool? E.
Double Check
1. Is a double check of data being input is carried out to ensure its correctness? 2. If yes, to what extent and whether this is being complied with in actual practice? Also are the results documented? F.
Control Totals as an input control
1. What all control totals exist in the system, if any?
Page 5 of 10
.
2. For each of the control totals, try tr y to investigate the nature of this control total 3. Did this control total exist in i n the manual system as well? 4. Who posts the control total, at what time (before the data entry or after) or is it system generated? 5. Are there occasions when the control total differs from the total of amount posted? 6. How, in such cases, is the total reconciled? G.
Any other form of input control and ensuring accountability?
1. Look for macro level indicators for data accuracy or lack of it, e.g. unreliable reports. 2. How are the data entry mistakes rectified? Is another input sheet prepared? 3. Are such modifications documented? 4. When previously recorded input is corrected, does the system retain previous values (for archiving purposes) as well as the current value? 5. Does a log exist indicating who had input the data and who made corrections? 6. Are the input corrections documented and properly authorized? H.
General
1. Are common input mistakes kept note of and circulated to all staff? 2. If someone intentionally wants to feed incorrect input, can it be prevented/ detected? 3. Does someone certify that all input has been fed completely and correctly? 4. Authorization of Input and completeness of input 5. Is the procedure for authorizing input and input corrections fully documented? 6. What are the controls to ensure completeness of input? 7.
Data Safety and Reliability
Data must be safe and secure at all times, all changes / modifications to data (including master data) must be documented and properly authorized. In a good system, it should be possible to link a particular entry (initial or modification) to the person who made that entry. This requires keeping of proper logs and existence of a system of review of these logs. These provide a good and effective audit trail. The physical access and logical access controls, authorizations to use the system (complete or partial) or lack of it, hardware security (ports, access to ports to outsiders etc.), net security, LAN security etc. could be the concern areas. Security becomes more important in a client-server environment and for distributed systems. Many times, back-end security is compromised, thus making the system totally unreliable.
Page 6 of 10
.
A.
IS Security
1. Has an officer been given overall responsibility of the IS security? 2. Are the users made aware of the IS Security requirements r equirements on a frequent basis? 3. Is there a system of recording and reporting security incidents? 4. What all audit trails are available in the system? 5. If it is a client-server environment, are back-end adjustments possible? 6. How many persons are authorized to carry out back-end adjustments? 7. Is the process of back-end adjustments fully documented and are logs available? 8. Has the office classified data based on its criticality? 9. Is there a data custodian, who ensures data integrity at all ti mes? 10. Are audit trails available along each path to ensure location of deviant activity? 11. Is the system susceptible to backend adjustments adjustments i.e., accessing database or programme not through the t he application? 12. Is yes, does adequate supervisory control exist over backend adjustments? 13. List transactions that are done backend 14. Are logs maintained of individual in dividual activities? 15. Can the system uniquely identify each event, transaction or activity on the system to a particular individual? 16. Are passwords periodically changed? 17. Does the IS security manager or similar appropriate officer periodically issue circulars, guidance etc for good IS practices? 18. Are the users also using internet i nternet through the same system? 19. Are the network security controls e.g. routers, firewalls etc in place? 20. Does review of exceptions take place periodically? Is there a formal process available to follow up the exceptions? 21. Are only needed drives active? Is there a control on the use of the disk drives and copying of the contents from the computer? B.
Change Control Procedures
1. Is there adequate change control procedure in place to ensure that the software is not modified unauthorisedly? unauthorisedly? 2. In case of modifications, are these recorded and properly approved? 3. How is the need for such modifications assessed? 4. In distributed systems, is the same version running on all machines? C.
Virus Protection
1. Is a security drill prescribed and is in use to ensure that the system is protected from virus etc. from the net? 2. Is there a formal and documented method for reporting security incidents and their escalation? D.
Storage Controls
Page 7 of 10
.
Integrity of data stored 1. How many persons are authorized to modify the stored data at a later date? 2. Is there a log kept and is available to assess the extent and frequency of changes/modification changes/modification carried out in the recorded data? 3. Is an input sheet or any document prepared and authorized by someone before any modification is carried out? 4. Is there a prescribed procedure for such modifications? 5. Has someone been appointed as the “custodian” of data and has been given the overall responsibility of ensuring integrity of stored data? 6. Has the data and the resulting information been classified as sensitive, confidential, important etc. with a view to determine the security levels required by each group of data? 7. Was an IS security audit ever carried out? 8. Is a hot copy of the database taken daily and the transaction files maintained to rebuilt the database up to the current point in case of loss of data 9. Is the back up kept in a location outside the office? 10. Are backups periodically tested? 11. Is the database tested periodically for consistency? 12. Is record of testing t esting database and backups maintained? 13. Is a master copy of the software maintained securely with designated official? 14. Is the compiled version of software compared with that of master copy so that the same version is running? 15. Is the access control matrix periodically reviewed to keep it up to date? 16. Is the storage medium used to maintain live data and backup periodically tested for defects? 17. Are access rights to system s ystem based on formally approved Access Control matrix? 18. Are damaged storage devices (medium), e.g. hard disk destroyed appropriately? 19. Are backups sent for safe custody? E.
Logs
1. Please list all logs which are being kept by the s ystem? 2. Are all these logs printed and reviewed by any authorized person and necessary action initiated? 3. Is this process properly documented? F.
Transmission controls
1. Is there a network administrator, authorized to give away rights to various users? 2. Are different users classified and given usage rights accordingly? 3. Is there a document available indicating such classification?
Page 8 of 10
.
G.
Processing Controls
Auditing process control may require use of test data or the parallel simulation techniques. However, However, the following foll owing may be looked into even if these techniques are not used. 1. Is there some process controls built in system enabling exception reporting (e.g. any abnormal transaction, any inconsistent entry, any repeated irregularity etc?) 2. Are various rules etc. correctly mapped in the system? 3. While processing the data, does the system s ystem compare results with some benchmarking (some sort of validation of results), like budget papers etc.? 4. Concurrency controls: Are proper controls in place to ensure that only one user at a time can modify the t he database? 5. In case of abrupt disruption disrupti on while processing data, does the system goes back to the State it was before the processing of current transaction was taken up to ensure atomicity of the transaction? 6. Are run to run control totals maintained to ensure completeness completeness of processing? 7. Do designated supervisory controls controls exist to verify and confirm that the processing is complete? 8. Was specifically generated test data ever run on the system to cover each of the processing logic paths? 9. If yes, has the testing through test data resulted in adverse observations? 10. Has the processing been tested through parallel run? 11. If yes, have the results of the parallel run at the final round of testing gave completely satisfactory results? 12. Are the normal processing time metrics (parameters) defined or usually understood? 13. If yes, is the actual performance over the period of time matching with the normal targeted performance as regards systems response time etc? 14. Is audit trail built in to ensure that processing and the changes to various tables can be traced with certainty? 15. Is there a record (manual or electronic) indicating the processing errors that would have cropped in while live processing? 16. If yes, are such exceptions appropriately handled? 17. Is manual reconciliation of the results of the processing done occasionally to ensure that the system is functioning as envisaged? 18. Have the integrity constraints like relational integrity and referential integrity in the database appropriately tested and found satisfactory? This will be enabled by data inconsistencies through data analysis. 8.
Output controls
1. Are procedures in place to ensure that only authorized persons can generate the reports from system 2. Is the system capable of generating output that can be emailed or transferred otherwise through the internet? 3. Is the output distributed only to authorized persons?
Page 9 of 10
.
9.
Feedback
1. Is there a mechanism to elicit formal feedback from users making data entry and also from other users and customers? 2. Look for major achievements as well as major weaknesses or limitations 3. Pl indicate any unique practices/work methods that have been generated 4. Is there any need for up gradation of software? 5. Are there any limitations of hardware, personnel or processes in system? 6. Is the training provided and competence levels of system officials are commensurate with the work they perform? 7. Identify and investigate serious risks to security of data, hardware and software of system s ystem package? 8. Look at the grievance/ complaint registers to ascertain the effectiveness of the grievance/ complaint redressal mechanism. This should be looked at from the point of view of the users(DEO/ Manager etc in the organisation) and the ultimate beneficiaries (customer/citizen) of the t he system. 10.
Other parameters
Obtain and study the data structure for possible comments – issues like normalization etc. System design can also be studied Can the system be replicated on a stand alone computer enabling running of test-data etc.?
Page 10 of 10
