Microsoft is considering moving to Rust; potential opportunity for Ada2020

Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Optikos]

https://www.zdnet.com/article/microsoft-to-explore-using-rust

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Maciej Sobczak]

> https://www.zdnet.com/article/microsoft-to-explore-using-rust

"The OS maker has been looking for safer C and C++ alternatives for years."

And? What makes the current context so much different, to enable the change this time?

Certainly, now there are even more legacy apps than years ago, thus making the potential change even more difficult than it was already when they decided not to change.

What I find the most interesting in this article, however, is that apparently C# did not make it as a preferred language in the Microsoft ecosystem. Not even after all the billions thrown into its marketing. This makes me even more skeptical about alternatives.

-- 
Maciej Sobczak * http://www.inspirel.com

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Optikos]

On Wednesday, July 17, 2019 at 1:53:26 PM UTC-5, Maciej Sobczak wrote:
> > https://www.zdnet.com/article/microsoft-to-explore-using-rust
> 
> "The OS maker has been looking for safer C and C++ alternatives for years."
> 
> And? What makes the current context so much different, to enable the change this time?

The advent of Xlang begat from the minor miracle that was C++/WinRT 1.0.  Kenny Kerr's C++/WinRT was what (in reverse-chronological order) Herb Sutter's C++/CX, Herb Sutter's C++/CLI, and Stan Lippman's Managed C++ was trying to achieve but failed (3 times!) to grasp Kenny Kerr's correct vision for using C++'s types to model COM's types in the least-obnoxious least-culture-clash way.
https://kennykerr.ca/2018/10/10/xlang

> Certainly, now there are even more legacy apps than years ago, thus making the potential change even
> more difficult than it was already when they decided not to change.

Microsoft's current strategy seems to be to wrap legacy infrastructure via xlang (nee C++/WinRT now repurposed into being multilingual e.g. Python).  Xlang (which is currently at the heart of C++/WinRT 2.0) could be a key wrap all-prior-eras-of-software strategy of incrementally converting to more & more Rust in a Rust-wrap-now-but-fatally-suffocate-C/C++/C#-later strategy to removing legacy C/C++/C# code over coming years, as it is converted opportunistically from guts within unsafe-Rust-wrapper to pure-Rust guts (perhaps still with a little obligatory unsafe at the core but Rust Rust Rust all the way).

And the article does emphasize that Rust is merely under consideration.  This Xlang-based surround-&-suffocate-over-time stragegy could be followed by, say, C#-with-Rustesque-borrow-checker instead.  I can hear it now:  ‘We seriously considered Rust, but in the end we went with something a little bit different instead.’

> What I find the most interesting in this article, however, is that apparently C# did not make it as a
> preferred language in the Microsoft ecosystem. Not even after all the billions thrown into its marketing. 

Yes, I also was wondering why Microsoft isn't fervently exploring adding a Rust-esque borrow checker to C#, which would be the natural Microsofty thing to do, at least in prior eras of Microsoft.

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Paul Rubin]

Optikos <optikos@verizon.net> writes:
> Yes, I also was wondering why Microsoft isn't fervently exploring
> adding a Rust-esque borrow checker to C#

C# is garbage collected, I thought.  No need for anything like a borrow
checker.

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Optikos]

On Wednesday, July 17, 2019 at 6:35:12 PM UTC-5, Paul Rubin wrote:
> Optikos writes:
> > Yes, I also was wondering why Microsoft isn't fervently exploring
> > adding a Rust-esque borrow checker to C#
> 
> C# is garbage collected, I thought.  No need for anything like a borrow
> checker.

Yes, historically C# is automatic-garbage-collected (AGC), but, to serve as a successor to C & C++ in the development of OS kernel and high-performance services/infrastructure at user-space at Microsoft (or elsewhere), C# would need to trim back any .NET excesses.  AGC is widely perceived (outside of the .NET and JVM communities) as one of those bloated excesses in .NET and in JVM worlds.  One could imagine C# becoming yet another language where AGC is optional, such as via different syntax for introducing AGCed versus borrow-checked modes of operation for each subroutine or each coroutine, operating in the programmer's choice of native or managed (to use Microsoft's terms) or unboxed or boxed (to use other communities' terms), much like C++/WinRT does. (Although in C++/WinRT, the “different syntax” is 1) normal C++ syntax for native versus 2) C++/WinRT's language-projection template(-header-only-library) for managed-via-ATL-to-COM.

Just as an Xlang projection of Rust can wrap C/C++ unsafe legacy code, an Ada language projection in Xlang can bring all that to Ada too, where Ada could also be a memory-safe wrapper language that permits opportunistic rewrite of more & more unsafe guts in the everless-wrapper-veneer evermore-deeper-to-the-core Ada layer.  The Ada language projection in Xlang would mimic the Xlang support for other languages (e.g., Python, Go, Rust) that resemble Ada to greater or lesser degrees.  Indeed, Ada2020 could conceivably most closely mimic Rust support in open-source Xlang to be a fast follower of this Rust experimentation at Microsoft—or at least mimicking Rust support in Xlang closer than any of the further-afield languages such as Python.  Conversely, adding Ada language projections via Xlang would permit Ada to take advantage of the vast forest of Python libraries by invoking them via the Xlang language projection.  Xlang is open-source at GitHub:
https://github.com/Microsoft/xlang

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Simon Wright]

Paul Rubin <no.email@nospam.invalid> writes:

> Optikos <optikos@verizon.net> writes:
>> Yes, I also was wondering why Microsoft isn't fervently exploring
>> adding a Rust-esque borrow checker to C#
>
> C# is garbage collected, I thought.  No need for anything like a
> borrow checker.

GC is about safely disposing of memory that's no longer accessible from
anywhere in the program.

Borrow checking (AIUI) is about making sure that only one part of the
program has write access to a particular piece of memory at once.

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Paul Rubin]

Simon Wright <simon@pushface.org> writes:
> Borrow checking (AIUI) is about making sure that only one part of the
> program has write access to a particular piece of memory at once.

Oh ok, for concurrency, good point.  Java uses "synchronized" for that
but it's not statically checked.

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Alejandro R. Mosteo]

On 20/7/19 0:07, Simon Wright wrote:
> Paul Rubin <no.email@nospam.invalid> writes:
> 
>> Optikos <optikos@verizon.net> writes:
>>> Yes, I also was wondering why Microsoft isn't fervently exploring
>>> adding a Rust-esque borrow checker to C#
>>
>> C# is garbage collected, I thought.  No need for anything like a
>> borrow checker.
> 
> GC is about safely disposing of memory that's no longer accessible from
> anywhere in the program.
> 
> Borrow checking (AIUI) is about making sure that only one part of the
> program has write access to a particular piece of memory at once.

Yes and no. If you check the first chapters in the Rust tutorial, heap 
memory reclamation is also performed deterministically when the borrow 
checker determines some memory has no more references left pointing to it.

I've seen it described as static GC. I guess it's a side-effect of the 
borrow checker; if you have to know how many mutable/immutable references 
exist to a block of memory, you end knowing when none are left.

That's my superficial understanding (haven't done any serious Rust). I'm 
frankly impressed by the concept. It seems to be a real contribution to 
the general landscape. Too bad the syntax gets ugly pretty quickly; I 
wonder about long-term maintainability.

Also, the particular interest of Microsoft in Rust is clear to me; it 
precludes whole classes of memory errors that make for a big chunk of 
security vulnerabilities.

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Brad Moore]

On Tuesday, August 6, 2019 at 11:01:14 AM UTC-6, Alejandro R. Mosteo wrote:
> On 20/7/19 0:07, Simon Wright wrote:
> > Paul Rubin <no.email@nospam.invalid> writes:
> > 
> >> Optikos <optikos@verizon.net> writes:
> >>> Yes, I also was wondering why Microsoft isn't fervently exploring
> >>> adding a Rust-esque borrow checker to C#
> >>
> >> C# is garbage collected, I thought.  No need for anything like a
> >> borrow checker.
> > 
> > GC is about safely disposing of memory that's no longer accessible from
> > anywhere in the program.
> > 
> > Borrow checking (AIUI) is about making sure that only one part of the
> > program has write access to a particular piece of memory at once.
> 
> Yes and no. If you check the first chapters in the Rust tutorial, heap 
> memory reclamation is also performed deterministically when the borrow 
> checker determines some memory has no more references left pointing to it.
> 
> I've seen it described as static GC. I guess it's a side-effect of the 
> borrow checker; if you have to know how many mutable/immutable references 
> exist to a block of memory, you end knowing when none are left.
> 
> That's my superficial understanding (haven't done any serious Rust). I'm 
> frankly impressed by the concept. It seems to be a real contribution to 
> the general landscape. Too bad the syntax gets ugly pretty quickly; I 
> wonder about long-term maintainability.
> 
> Also, the particular interest of Microsoft in Rust is clear to me; it 
> precludes whole classes of memory errors that make for a big chunk of 
> security vulnerabilities.

I believe it is also not just related to concurrency. For example, if you pass a pointer to an object into a function which deletes the object, the compiler will detect that use of that pointer after calling the function is not allowed.

Brad

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Alejandro R. Mosteo]

On 6/8/19 19:49, Brad Moore wrote:

> I believe it is also not just related to concurrency. For example, if you pass a pointer to an object into a function which deletes the object, the compiler will detect that use of that pointer after calling the function is not allowed.

Yes; concurrency safety (in a limited sense) is the side-effect, not the 
main point, I'd say.

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Jere]

On Wednesday, August 7, 2019 at 5:09:06 AM UTC-4, Alejandro R. Mosteo wrote:
> On 6/8/19 19:49, Brad Moore wrote:
> 
> > I believe it is also not just related to concurrency. For example, if you pass a pointer to an object into a function which deletes the object, the compiler will detect that use of that pointer after calling the function is not allowed.
> 
> Yes; concurrency safety (in a limited sense) is the side-effect, not the 
> main point, I'd say.

I think at some point before the language was first stabilized (2015), 
both were actively pursued.  Here are some musings from the original main 
developer back in 2013ish [1].  Later on, another Mozilla developer also
talked about a similar topic [2].

[1]: http://smallcultfollowing.com/babysteps/blog/2013/06/11/on-the-connection-between-memory-management-and-data-race-freedom/
[2]: https://manishearth.github.io/blog/2015/05/17/the-problem-with-shared-mutability/

It is interesting that both of them kind of hint at the idea of a large
single threaded program having similar challenges to a multithreaded
program, at least when considering how undefined behavior, data
invalidation, and data races occur.

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Alejandro R. Mosteo]

On 8/8/19 4:13, Jere wrote:
> On Wednesday, August 7, 2019 at 5:09:06 AM UTC-4, Alejandro R. Mosteo wrote:
>> On 6/8/19 19:49, Brad Moore wrote:
>>
>>> I believe it is also not just related to concurrency. For example, if you pass a pointer to an object into a function which deletes the object, the compiler will detect that use of that pointer after calling the function is not allowed.
>>
>> Yes; concurrency safety (in a limited sense) is the side-effect, not the
>> main point, I'd say.
> 
> I think at some point before the language was first stabilized (2015),
> both were actively pursued.  Here are some musings from the original main
> developer back in 2013ish [1].  Later on, another Mozilla developer also
> talked about a similar topic [2].
> 
> [1]: http://smallcultfollowing.com/babysteps/blog/2013/06/11/on-the-connection-between-memory-management-and-data-race-freedom/
> [2]: https://manishearth.github.io/blog/2015/05/17/the-problem-with-shared-mutability/
> 
> It is interesting that both of them kind of hint at the idea of a large
> single threaded program having similar challenges to a multithreaded
> program, at least when considering how undefined behavior, data
> invalidation, and data races occur.

I understand better now; thanks for these references.

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Optikos]

On Tuesday, August 6, 2019 at 12:01:14 PM UTC-5, Alejandro R. Mosteo wrote:
> On 20/7/19 0:07, Simon Wright wrote:
> > Paul Rubin <no.email@nospam.invalid> writes:
> > 
> >> Optikos writes:
> >>> Yes, I also was wondering why Microsoft isn't fervently exploring
> >>> adding a Rust-esque borrow checker to C#
> >>
> >> C# is garbage collected, I thought.  No need for anything like a
> >> borrow checker.
> > 
> > GC is about safely disposing of memory that's no longer accessible from
> > anywhere in the program.
> > 
> > Borrow checking (AIUI) is about making sure that only one part of the
> > program has write access to a particular piece of memory at once.
> 
> Yes and no. If you check the first chapters in the Rust tutorial, heap 
> memory reclamation is also performed deterministically when the borrow 
> checker determines some memory has no more references left pointing to it.
> 
> I've seen it described as static GC. I guess it's a side-effect of the 
> borrow checker; if you have to know how many mutable/immutable references 
> exist to a block of memory, you end knowing when none are left.

I suspect that each commenter prior to Alejandro Mosteo had their own variant definition of garbage collection.  Some people think garbage collection is limited to mark-&-sweep garbage collection, excluding reference counting.  Other people designate that definition as AGC (automatic garbage collection) and also include reference counting in the generic term garbage collection (without the automatic on the front).  Alejandro is correct:  if a technique can assure that the mutable count is not greater than one, then the technique has enough wherewithal to know when the mutable + immutable count goes less than 1 (i.e., zero) to release the resource via reference counting.  Although, how does Rust's borrow checker assure the lack of cycles (or assure that the cyclic references are self-contained in a glob that itself has an acyclic reference count, so that the entire glob is condemned en masse)?  How much of a borrow-checker-esque approach is going to appear in Ada202X, especially now that the timeline is extended?

The book _Garbage collection:  Algorithms for automatic dynamic memory management_ presents both of these variant definitions (and more shade-of-meaning flavors too).

https://www.worldcat.org/title/garbage-collection-algorithms-for-automatic-dynamic-memory-management/oclc/301048851

> That's my superficial understanding (haven't done any serious Rust). I'm 
> frankly impressed by the concept. It seems to be a real contribution to 
> the general landscape. Too bad the syntax gets ugly pretty quickly; I 
> wonder about long-term maintainability.
> 
> Also, the particular interest of Microsoft in Rust is clear to me; it 
> precludes whole classes of memory errors that make for a big chunk of 
> security vulnerabilities.

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Brad Moore]

On Tuesday, August 6, 2019 at 11:56:06 AM UTC-6, Optikos wrote:
> How much of a borrow-checker-esque approach is going to appear in Ada202X, especially now that the timeline is extended?

Likely we are looking at just what is provided by AI12-240,
see 
  http://www.ada-auth.org/cgi-bin/cvsweb.cgi/ai12s/ai12-0240-6.txt?rev=1.7

which is to provide a low-level minimal framework that can be built upon to provide more elaborate ownership based solutions. That is, we aren't at a point yet to provide a complete solution, and we don't want to standardise something before we have more confidence that it is the right solution. But it is thought that AI12-240 provides a common ground that is general enough to be applicable to whatever solution we might eventually end up with.

Brad

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Alejandro R. Mosteo]

On 6/8/19 19:56, Optikos wrote:

> Although, how does Rust's borrow checker assure the lack of cycles (or assure that the cyclic references are self-contained in a glob that itself has an acyclic reference count, so that the entire glob is condemned en masse)?  

It seems you are on your own (e.g. use weak references) to deal with these:

https://doc.rust-lang.org/book/ch15-06-reference-cycles.html

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[G. B.]

Optikos <optikos@verizon.net> wrote:
> https://www.zdnet.com/article/microsoft-to-explore-using-rust
> 


Microsoft, by nature, will explore the potential of everything X for which
there seems to be a market, provided that

- X is not controlled by government 
- Microsoft can have a say in the development of X

It about ROI.

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Shark8]

On Thursday, July 25, 2019 at 12:40:13 PM UTC-6, G. B. wrote:
> Optikos wrote:
> > https://www.zdnet.com/article/microsoft-to-explore-using-rust
> > 
> 
> 
> Microsoft, by nature, will explore the potential of everything X for which
> there seems to be a market, provided that
> 
> - X is not controlled by government 
> - Microsoft can have a say in the development of X
> 
> It about ROI.

Microsoft certainly *could* have a say in the development of Ada, just join the ARG.

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Norman Worth]

Shark8 wrote:
> On Thursday, July 25, 2019 at 12:40:13 PM UTC-6, G. B. wrote:
>> Optikos wrote:
>>> https://www.zdnet.com/article/microsoft-to-explore-using-rust
>>>
>>
>>
>> Microsoft, by nature, will explore the potential of everything X for which
>> there seems to be a market, provided that
>>
>> - X is not controlled by government
>> - Microsoft can have a say in the development of X
>>
>> It about ROI.
> 
> Microsoft certainly *could* have a say in the development of Ada, just join the ARG.
> 
About the time Ada83 was standardized, Microsoft emphatically declared 
that it would never (never!) support or have anything to do with Ada.

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Mehdi Saada]

> About the time Ada83 was standardized, Microsoft emphatically declared 
> that it would never (never!) support or have anything to do with Ada.
Really ? What the heck happened with them ? How come did they start harboring such hate?

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Shark8]

On Tuesday, October 1, 2019 at 2:04:07 AM UTC-6, Mehdi Saada wrote:
> > About the time Ada83 was standardized, Microsoft emphatically declared 
> > that it would never (never!) support or have anything to do with Ada.
> Really ? What the heck happened with them ? How come did they start harboring such hate?

I don't know; but some years ago I had the fortune of talking to someone who had been involved in a company doing an evaluation/audit of the Windows code -- around version 2, or perhaps 3.1 (before Windows for Workgroups 3.11) -- and he related to me that his company recommended a rewrite in Ada.

Of course Microsoft didn't follow the recommendation, but it's interesting that they had the recommendation made to them.

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Shark8]

On Wednesday, July 17, 2019 at 12:17:45 PM UTC-6, Optikos wrote:
> https://www.zdnet.com/article/microsoft-to-explore-using-rust

Of possible interest is this discussion on xlang:
https://github.com/Microsoft/xlang/issues/226

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Optikos]

On Friday, July 26, 2019 at 9:11:33 AM UTC-5, Shark8 wrote:
> On Wednesday, July 17, 2019 at 12:17:45 PM UTC-6, Optikos wrote:
> > https://www.zdnet.com/article/microsoft-to-explore-using-rust
> 
> Of possible interest is this discussion on xlang:
> https://github.com/Microsoft/xlang/issues/226

Shark8, thank you for engaging Microsoft on this topic; many valid & useful critiques were raised.  16 days ago, it was closed due to the gist of the replies being philosophical design.  I would recommend shifting strategies to the following:
1) Find concrete examples of where slang's current approach to types will misbehave or be less than fully useful or, best yet, be a total unmitigated disaster.
2) Aim straight for the jugular vein in as tight of language as possible.  Give concrete ugliest vile examples in real code snippets in slang's favorite top-priority programming languages (e.g., Python, Javascript, C++).  The examples should be nasty push-comes-to-shove total embarrassments of the current approach to types in xlang interior between languages (and library APIs thereof).
3) Despite #2 being the worst bad news that we can find, #2's wording should always be polite, short & to-the-point, and service-with-a-smile, emphasizing the wow-factor of in the tone of: “Gee, that looks like a bad situation that you & your type system got yourself into; how are you going to get out of •that• tight spot without Ada's subtype capabilities?  Good luck with that!  You realize that the solution to this problem's path is already blazed in Ada's wise subtype system, don't you?  Of course, we could stand on the shoulders of Ada-giants that came before decades ago, instead of trying to reinvent the square-wheel badly.”  But not in that direct wording—more polite & oblique than that, but with the same import & meaning.
4) Enter each example as a new defect-issue number, instead of a rambling litany-list of minor issues.  Make xlang's leadership sweat each defect as a stand-alone separate mess in a review meeting with their peers & bosses, rather than simply deftly closing the issue as off-topic “design”.

Re: Microsoft is considering moving to Rust; potential opportunity for Ada2020

[Optikos]

On Saturday, July 27, 2019 at 3:03:50 PM UTC-5, Optikos wrote:
> On Friday, July 26, 2019 at 9:11:33 AM UTC-5, Shark8 wrote:
> > On Wednesday, July 17, 2019 at 12:17:45 PM UTC-6, Optikos wrote:
> > > https://www.zdnet.com/article/microsoft-to-explore-using-rust
> > 
> > Of possible interest is this discussion on xlang:
> > https://github.com/Microsoft/xlang/issues/226
> 
> Shark8, thank you for engaging Microsoft on this topic; many valid & useful critiques were raised.  16 days ago, it was closed due to the gist of the replies being philosophical design.  I would recommend shifting strategies to the following:
> 1) Find concrete examples of where slang's current approach to types will misbehave or be less than fully useful or, best yet, be a total unmitigated disaster.
> 2) Aim straight for the jugular vein in as tight of language as possible.  Give concrete ugliest vile examples in real code snippets in slang's favorite top-priority programming languages (e.g., Python, Javascript, C++).  The examples should be nasty push-comes-to-shove total embarrassments of the current approach to types in xlang interior between languages (and library APIs thereof).
> 3) Despite #2 being the worst bad news that we can find, #2's wording should always be polite, short & to-the-point, and service-with-a-smile, emphasizing the wow-factor of in the tone of: “Gee, that looks like a bad situation that you & your type system got yourself into; how are you going to get out of •that• tight spot without Ada's subtype capabilities?  Good luck with that!  You realize that the solution to this problem's path is already blazed in Ada's wise subtype system, don't you?  Of course, we could stand on the shoulders of Ada-giants that came before decades ago, instead of trying to reinvent the square-wheel badly.”  But not in that direct wording—more polite & oblique than that, but with the same import & meaning.
> 4) Enter each example as a new defect-issue number, instead of a rambling litany-list of minor issues.  Make xlang's leadership sweat each defect as a stand-alone separate mess in a review meeting with their peers & bosses, rather than simply deftly closing the issue as off-topic “design”.

Dang autocorrect!  xlang not slang above in a few places.

5) Fork a few of xlang's individual work-products here & there (e.g., a design note) to show them as one-offs what the Ada-esque better way would look like concretely, precisely.  If Microsoft fails to follow Ada's wiser lead in xlang's type system, then really fork all of xlang to create an Ada-centric better way of interoping with all the other mainstream languages.