Re: Ada vs. C Comparison Data ?

[cis.ohio-state.edu!magnus.acs.ohio-state.edu!csn!arrayb!wicklund@ucbvax.B erkeley.EDU (Tom Wicklund)]

In <1991Aug08.083832.21573@netcom.COM> jls@netcom.COM (Jim Showalter) writes:

>wicklund@intellistor.com (Tom Wicklund) writes:

>>Of course, this is like sugar companies showing health risks in
>>artificial sweeteners, tobacco companies showing no health risks for
>>cigarettes, etc.  The Air Force has basically been ordered to find no
>>exceptions for ADA use, so it's no surprise that they have.

>Funny, I don't recall any such wording in the charter for the group
>the did the study. Must have been in the fine print somewhere. The
>charter I read seemed pretty straightforward: compare the two languages
>and see if there is any compelling reason to weaken the Ada mandate in
>favor of C++ for at least some classes of applications. Nothing at all
>about finding no exceptions to Ada use, whitewashing, or any of the rest
>of it.

My posting was written before reading the press conference remarks
posted here.  Below are some comments after reading them, some of
which will be pertinent to a non-defense industry comparison of Ada
and C++.  The quotes below are from the press conference remarks
posted here.


>>I imagine ATT could sponsor a study and find the opposite result.

>Not if they were honest about it.

Actually, if ATT sponsored an objective comparison of C and Ada they
would probably come up with the opposite result.  The reasons would be
the same reasons that DOD prefers Ada -- ATT has a strong committment
to C, there are more tools, compilers, etc. for C on ATT machines, ATT
has C training programs but not Ada, etc.  To a large extent, the Air
Force's reasons for Ada are because they are Ada oriented.



Some comments on the summary of the Air Force comparison of Ada and C++:

PLEASE NOTE: I'm not disputing the results of the study in the context
of DOD use of Ada.  However, this subject was brought up in the
context of a non DOD user comparing Ada and C.  Many of the points I
make below are to point out this (necessary) bias in the study which
may not apply for non-DOD users.

After reading the press conference remarks, I agree that this was not
a whitewash.  However, the fact remains that the study was performed
by groups with a vested interest in Ada and DOD.  DOD has a well
documented history of adjusting results to match what they want to
see, whether it's overestimates of Soviet capabilities or test results
of weapons.  A study of this sort must be carefully examined for the
assumptions used and checked for any intentional or unintentional
bias.  A defense contractor is also likely to think twice before
slamming Ada since they've got a lot invested in it.


>A.   Tools, Environments, and Training: IDA Substudy.

>     The Institute for Defense Analyses (IDA) collected and
>analyzed information on the market availability of commercial-
>off-the-shelf products available from U.S. sources for Ada and
>C++ compilers, tools, education, and training.

Read this section carefully -- several justifications for Ada are
because DOD has a strong investment in Ada.  This is a valid reason
for DOD preferring Ada but not necessarily a good reason for a
commercial firm without an existing Ada investment.  Other reasons are
based on C++ being younger, unstandardized, and not currently under
DOD control.  This says nothing about the capabilities of the
language, just that Ada is already in the DOD process.  A good reason
for DOD, maybe not for somebody else.

>     Both languages are supported on PCs and workstations.  Ada
>is also supported on mainframes.  Ada, but not C++, has cross
>compilation systems.

Untrue.  I'm sure cfront is available on mainframes running UNIX.
Cross compilation systems include at least Microtek C++ for the 68K
and Turbo C++ with Paradigm Systems embedded development package.  I'm
sure others exist.  The Turbo C++ add-on has existed for over a year,
so should have been known to IDA if they did a thorough study.  C++
tools aren't oriented to DOD environments, but this is because there's
no market due to the Ada requirement.

>Code generators exist for Ada but none so far for C++.

Untrue -- G++, Zortech C++, Borland C++, Oregon C++, Green Hills C++,
NDP C++, and TopSpeed C++ are listed as native compilers in the C++
products list periodically posted to comp.lang.c++.  A couple might
actually be cfront based, most are not.


>B.   Faceted IBM Language Selection Methodology: SEI Substudy.

To evaluate these results, look at the detailed report.  Different
applications have very different requirements.  Somebody else might
weigh the categories differently.


>C.   Macro Cost Analysis: CTA Substudy.

I don't know enough statistics, but the relatively small number of C++
projects and the relatively small difference in the numbers may make
some of this study's results meaningless.

>                           Productivity    Number of
>                           (SLOC/MM)       Data Points

See comp.software-eng for a discussion of whether measuring
productivity by lines of code per man month is a valid test.  these
results also don't consider the relative functionality of a line of
Ada or C++ (I have no idea what it would be).

>Typically, the Ada developments were performed in accordance with
>military standards and incorporated formal reviews, additional
>documentation, and additional engineering support activities such
>as formal quality assurance (QA) and configuration management
>(CM).  Most C++ projects are commercial and do not extensively
>incorporate such activities.  Additionally, on such projects
>developers are typically intimately involved with users,
>resulting in considerably less requirements engineering effort. 
>Consequently, applications on which C++ is used are inherently
>less costly, so that the reported productivity rates are
>favorably skewed toward C++.

Funny -- Software quality experts will tell you that all the formal
reviews, QA, etc. improve productivity and reduce costs.  Informal
requirements with frequent user interaction are especially expensive
compared to a fixed set of requirements up front.  So if the C++
projects were done more formally they should cost less, right?

>                         Integration    FQT            Number of
>                         Error Rates    Error Rates    Data      
>                         (Errors/KSLOC) (Errors/KSLOC) Points
> *   Norm (all languages) 33             3              543
> *   Average (Ada)        24             1              153
> *   Average (C++)        31             3               23

An apples and oranges comparison if Ada developments used better
design methods, so caught more problems before integration.  The C++
rate should go down with the application of these methods.

Again, this study points out the relative maturity of Ada, the tight
standardization, and DOD's single language strategy.  Important for
some users, not for others.


>D.   Micro Cost Analysis: TRW Substudy.

The summarized results of this study don't give enough information to
be meaningful.  I do note that some Ada advantages appear to be in the
standardized support environment, which one can get with C++ by
choosing the proper add-on product or enforcing some development
guidelines.

However, the primary strengths and weaknesses listed appear accurate.


>E.   Ada Policy Issues: NPS Study.

This summary appears well thought out, definately not a whitewash.
They acknowledge the usefulness of 4GLs and other higher level
software generators while keeping to DOD's Ada commitment.  Again, the
Ada poriton applies more to DOD than other users.

This report does skirt around an interesting issue -- The use of CASE
tools which directly generate code or languages such as Ada versions
of LEX, YACC, and state machine generators are technically violations
of the Ada mandate.  If I write a scanner in ALEX I'm not writing
strict Ada code.  Taking the Ada mandate literally one can't use any
of these tools since your source code isn't Ada.  Hopefully DOD will
find a way to handle tools of this sort without prohibiting them or
getting into a different type of language explosion.