x64 roadmap ?

Topics: Developing Cosmos (Working on Cosmos source code directly)
Developer
Sep 29, 2012 at 12:36 PM

Kudzu I hope your new SSD is up and running. Did you had time to make some planing on how we could move to x64 ? Did you already identified some tasks I could deal with ?

Coordinator
Sep 29, 2012 at 10:30 PM
On 9/29/2012 8:36 AM, BlueSkeye wrote:
> Kudzu I hope your new SSD is up and running. Did you had time to make
> some planing on how we could move to x64 ? Did you already identified
> some tasks I could deal with ?

Yes I posted last week. First step is that we need to convert the
compiler over to the new X# assembler.. then migration to x64 will be
much more straightforward.
Developer
Sep 30, 2012 at 9:43 AM

Apologize, I may not have been attentive enough. Do you remind where you posted ? I can't manage to retrieve your post neither in Codeplex discussions nor on the Yahoo group.

Coordinator
Sep 30, 2012 at 3:06 PM
On 9/30/2012 5:43 AM, BlueSkeye wrote:
> Apologize, I may not have been attentive enough. Do you remind where you
> posted ? I can't manage to retrieve your post neither in Codeplex
> discussions nor on the Yahoo group.

It was basically the same as I posted here.. :)

Take a look at the new X# compiler, and see how we can integrate it to
the compiler and work closely with me.... This is the first necessary
step to go x64 because right now the compiler has about 3 ways of doing
thing (partial ports) and it will be hectic for you , so easier to
homogenize first.
Coordinator
Sep 30, 2012 at 4:23 PM

When doing commits, please discuss first so we can see if anything goofs up. I know there's testing but compiler is hard to test and easy to break...

Op 30 sep. 2012 17:06 schreef "kudzu" <notifications@codeplex.com> het volgende:

From: kudzu

On 9/30/2012 5:43 AM, BlueSkeye wrote:
> Apologize, I may not have been attentive enough. Do you remind where you
> posted ? I can't manage to retrieve your post neither in Codeplex
> discussions nor on the Yahoo group.

It was basically the same as I posted here.. :)

Take a look at the new X# compiler, and see how we can integrate it to
the compiler and work closely with me.... This is the first necessary
step to go x64 because right now the compiler has about 3 ways of doing
thing (partial ports) and it will be hectic for you , so easier to
homogenize first.

Read the full discussion online.

To add a post to this discussion, reply to this email (Cosmos@discussions.codeplex.com)

To start a new discussion for this project, email Cosmos@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com

Coordinator
Sep 30, 2012 at 4:25 PM
On 9/30/2012 12:24 PM, mterwoord wrote:
> When doing commits, please discuss first so we can see if anything goofs
> up. I know there's testing but compiler is hard to test and easy to
> break...

And for X# we'll have to discuss before hand anyways how to integrate to
compiler.
Developer
Sep 30, 2012 at 10:30 PM

> Take a look at the new X# compiler

Can you explain where is the border between the "new" and "older" X# compiler (changeset number would be appreciated) ?

> and work closely with me....

Team leaders and their project history knowledge are always welcome. They're expected not to let us run once again with the same mistakes.

> right now the compiler has about 3 ways of doing thing (partial ports)

Does it mean at least two of them have to be dropped ...

> so easier to homogenize first.

or should we try to run after a catch-all solution ?

> and it will be hectic for you,

I'm a cold-blooded animal :-) Ok this is unfair. Your sentence has been reordered from your original post.

@mterwoord : "Tell before doing" is my daily motto to my team. I hope I know how to stick with it for myself. Please call me to orders with facts as soon as you feel I'm about to cross the line.

Coordinator
Oct 1, 2012 at 7:55 AM
Seriously though: I think you're serious enough for things, but it's just that compiler stuff is easy to goof up (you talked about history of project members? i goofed the project up quite a number of times..)

the "new" x# is everything using the .xs file extension i think. Chad: I think it still needs to be extended a bit so X# can be used as a real code generation engine, with parameters etc?




On Mon, Oct 1, 2012 at 12:30 AM, BlueSkeye <notifications@codeplex.com> wrote:

From: BlueSkeye

> Take a look at the new X# compiler

Can you explain where is the border between the "new" and "older" X# compiler (changeset number would be appreciated) ?


> and work closely with me....

Team leaders and their project history knowledge are always welcome. They're expected not to let us run once again with the same mistakes.


> right now the compiler has about 3 ways of doing thing (partial ports)

Does it mean at least two of them have to be dropped ...


> so easier to homogenize first.

or should we try to run after a catch-all solution ?


> and it will be hectic for you,

I'm a cold-blooded animal :-) Ok this is unfair. Your sentence has been reordered from your original post.

@mterwoord : "Tell before doing" is my daily motto to my team. I hope I know how to stick with it for myself. Please call me to orders with facts as soon as you feel I'm about to cross the line.

Read the full discussion online.

To add a post to this discussion, reply to this email (Cosmos@discussions.codeplex.com)

To start a new discussion for this project, email Cosmos@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com


Coordinator
Oct 3, 2012 at 2:16 PM
On 10/1/2012 3:56 AM, mterwoord wrote:
> Seriously though: I think you're serious enough for things, but it's
> just that compiler stuff is easy to goof up (you talked about history of
> project members? i goofed the project up quite a number of times..)

Should be able to compare the NASM files and do a diff. Small changes
might occur with syntax and formatting but as a first step, asm should
be identical....

> the "new" x# is everything using the .xs file extension i think. Chad: I
> think it still needs to be extended a bit so X# can be used as a real
> code generation engine, with parameters etc?

Yes a bit. But its all been planned for.
Coordinator
Oct 3, 2012 at 2:17 PM
On 9/30/2012 6:30 PM, BlueSkeye wrote:
> Can you explain where is the border between the "new" and "older" X#
> compiler (changeset number would be appreciated) ?

Sorry Ive been "missing" for a few days... I've got a house under
construction and last few days were very busy.

Hmm.. there really isn't a defined border. Each IL op has a class which
emits asm on its behalf. There are several methods to emit ASM which
have evolved over time. Some use the old X# (not many). Some use the
assembler we have, some use the assembler we have but use it to emit raw
text, etc...

The first step is to rewrite ALL of those classes to use the NEW X#
compiler. And there are some steps inside X# that use the assembler
which should be changed too.

So the way I see it:

Step 1 - The old assembler should be jettisoned. It was designed to emit
both bin or nasm, but usage of it is overly complex and I find myself
wanting to point a gun at my head everytime I have to use it as it takes
something like

MOV AX, 0x00 and requires me to write about 50x the code to emit that.
This and for various other reasons I think we should just go to straight
text emission since X# becomes our typed front end and the only
"assembler" usage is moved inside X#. If we ever do binary emission
directly again and bypass NASM there are a variety of approaches we can
take. But we should not worry about this for now at all nor make any
provisions as they sidetrack us and NASM is blazing fast. Thus binary
output is not needed for the foreseeable future and should not distract
or impact our code decisions at all at this point in time.

If you look in the X# compiler you will see I made a new simplistic
assembler class. This is the start of step 1 and should be continued.

Step 2 - Convert IL op classes in the compiler to use the new X#
compiler. Probably each class becomes an XS file.

To do this we need to add parameters to X# functions, but I already
planned to do this.

Step 3 - Purge old assembler and old X# compiler. I think in fact I
already purged the old X# compiler.. need to look.

Step 4 - Move to x64.

Step 5 - Onward towards world domination.


> Team leaders and their project history knowledge are always welcome.
> They're expected not to let us run once again with the same mistakes.

Contact me offline please and I'll work with you to coordinate this.

> > right now the compiler has about 3 ways of doing thing (partial ports)
>
> Does it mean at least two of them have to be dropped ...

Yes.

> > so easier to homogenize first.
>
> or should we try to run after a catch-all solution ?

X# does do it all. Its not a matter of capability, but of porting old code.
Developer
Oct 4, 2012 at 9:02 AM

Kudzu I carefully read your answer. Those steps make sense. I had some additional questions to make sure I understood your proposal the right way. Today we have :

- Cosmos.Assembler which for now contains one class per x86/x87/SSE instruction. These classes know how to emit NASM compliant source code.
- Cosmos.IL2CPU which contains one class per IL instruction. Those classes in turn instantiate classes from Cosmos.Assembler.
- Cosmos.Compiler.XSharp that is a translator from X# syntax to NASM source code.

Basically it looks like we have a layer (Cosmos.Assembler) that seems to add little value. I understand the Cosmos.Assembler assembly is what you name "The old assembler". So when you say "I think we should just go to straight text emission" should we understand that Cosmos.IL2CPU IL classes should directly emit source code and bypass the Cosmos.Assembler classes ? If so, should we emit directly NASM code or should we keep an additional intermediate layer and emit X# code ? One way or another, this means simple assembler instructions text such as the "MOV AX, 0x00" one you exemplified would be directly emitted by IL op classes from the Cosmos.IL2CPU assembly.

You're not talking about plugs. Some of them rely on Cosmos.Assembler for NASM source code emission. Do you intend them to be migrated the same way than IL classes above ?

The XShap.Nasm assembly and its Assembler class is what I understand to be the "new simplistic assembler class". It looks like you expect it to be extended with new methods as part of step 1. What kind of features do you expect to be added during this step ?

From my point of view step 1 and 2 looks very tightly coupled with each other and having them split into two steps looks a little bit artificial, unless additions to XSharp.Nasm.Assembler class is much more work than I can foresee.

Coordinator
Oct 4, 2012 at 9:07 AM
the IL classes should use X#, so X# is used for generating the text source, and can later on be converted to x64 (arm, etc).
Plugs should ideally be evaluated to see if they still are needed, and if so, modified to use plugs (or norma c# code where possible).

adding stuff to the new compiler should be done when needed.converting the IL classes to use (or to be) X# is going to take a bit of work i think..


On Thu, Oct 4, 2012 at 11:02 AM, BlueSkeye <notifications@codeplex.com> wrote:

From: BlueSkeye

Kudzu I carefully read your answer. Those steps make sense. I had some additional questions to make sure I understood your proposal the right way. Today we have :

- Cosmos.Assembler which for now contains one class per x86/x87/SSE instruction. These classes know how to emit NASM compliant source code.
- Cosmos.IL2CPU which contains one class per IL instruction. Those classes in turn instantiate classes from Cosmos.Assembler.
- Cosmos.Compiler.XSharp that is a translator from X# syntax to NASM source code.

Basically it looks like we have a layer (Cosmos.Assembler) that seems to add little value. I understand the Cosmos.Assembler assembly is what you name "The old assembler". So when you say "I think we should just go to straight text emission" should we understand that Cosmos.IL2CPU IL classes should directly emit source code and bypass the Cosmos.Assembler classes ? If so, should we emit directly NASM code or should we keep an additional intermediate layer and emit X# code ? One way or another, this means simple assembler instructions text such as the "MOV AX, 0x00" one you exemplified would be directly emitted by IL op classes from the Cosmos.IL2CPU assembly.

You're not talking about plugs. Some of them rely on Cosmos.Assembler for NASM source code emission. Do you intend them to be migrated the same way than IL classes above ?

The XShap.Nasm assembly and its Assembler class is what I understand to be the "new simplistic assembler class". It looks like you expect it to be extended with new methods as part of step 1. What kind of features do you expect to be added during this step ?

From my point of view step 1 and 2 looks very tightly coupled with each other and having them split into two steps looks a little bit artificial, unless additions to XSharp.Nasm.Assembler class is much more work than I can foresee.

Read the full discussion online.

To add a post to this discussion, reply to this email (Cosmos@discussions.codeplex.com)

To start a new discussion for this project, email Cosmos@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com


Coordinator
Oct 4, 2012 at 9:08 AM
Oh, and please try to use the yahoo mailinglist for development discussions like these.. :)

On Thu, Oct 4, 2012 at 11:07 AM, Matthijs ter Woord <matthijsterwoord@gmail.com> wrote:
the IL classes should use X#, so X# is used for generating the text source, and can later on be converted to x64 (arm, etc).
Plugs should ideally be evaluated to see if they still are needed, and if so, modified to use plugs (or norma c# code where possible).

adding stuff to the new compiler should be done when needed.converting the IL classes to use (or to be) X# is going to take a bit of work i think..



On Thu, Oct 4, 2012 at 11:02 AM, BlueSkeye <notifications@codeplex.com> wrote:

From: BlueSkeye

Kudzu I carefully read your answer. Those steps make sense. I had some additional questions to make sure I understood your proposal the right way. Today we have :

- Cosmos.Assembler which for now contains one class per x86/x87/SSE instruction. These classes know how to emit NASM compliant source code.
- Cosmos.IL2CPU which contains one class per IL instruction. Those classes in turn instantiate classes from Cosmos.Assembler.
- Cosmos.Compiler.XSharp that is a translator from X# syntax to NASM source code.

Basically it looks like we have a layer (Cosmos.Assembler) that seems to add little value. I understand the Cosmos.Assembler assembly is what you name "The old assembler". So when you say "I think we should just go to straight text emission" should we understand that Cosmos.IL2CPU IL classes should directly emit source code and bypass the Cosmos.Assembler classes ? If so, should we emit directly NASM code or should we keep an additional intermediate layer and emit X# code ? One way or another, this means simple assembler instructions text such as the "MOV AX, 0x00" one you exemplified would be directly emitted by IL op classes from the Cosmos.IL2CPU assembly.

You're not talking about plugs. Some of them rely on Cosmos.Assembler for NASM source code emission. Do you intend them to be migrated the same way than IL classes above ?

The XShap.Nasm assembly and its Assembler class is what I understand to be the "new simplistic assembler class". It looks like you expect it to be extended with new methods as part of step 1. What kind of features do you expect to be added during this step ?

From my point of view step 1 and 2 looks very tightly coupled with each other and having them split into two steps looks a little bit artificial, unless additions to XSharp.Nasm.Assembler class is much more work than I can foresee.

Read the full discussion online.

To add a post to this discussion, reply to this email (Cosmos@discussions.codeplex.com)

To start a new discussion for this project, email Cosmos@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com



Coordinator
Oct 4, 2012 at 11:31 AM

On 10/4/2012 5:08 AM, mterwoord wrote:
> the IL classes should use X#, so X# is used for generating the text
> source, and can later on be converted to x64 (arm, etc).
> Plugs should ideally be evaluated to see if they still are needed, and 
> if so, modified to use plugs (or norma c# code where possible).

The whole plug system needs overhauled and I have notes on that. I would avoid that for now with the exception of converting ASM plugs to use the new X# compiler.

Coordinator
Oct 4, 2012 at 12:00 PM
On 10/4/2012 5:02 AM, BlueSkeye wrote:
> - Cosmos.Assembler which for now contains one class per x86/x87/SSE
> instruction. These classes know how to emit NASM compliant source code.

This one will be eliminated.

> - Cosmos.IL2CPU which contains one class per IL instruction. Those
> classes in turn instantiate classes from Cosmos.Assembler.
> - Cosmos.Compiler.XSharp that is a translator from X# syntax to NASM
> source code.

These will be kept.

> Basically it looks like we have a layer (Cosmos.Assembler) that seems to
> add little value. I understand the Cosmos.Assembler assembly is what you
> name "The old assembler". So when you say "I think we should just go to

Yes.

> straight text emission" should we understand that Cosmos.IL2CPU IL
> classes should directly emit source code and bypass the Cosmos.Assembler

They should use XSharp.NASM instead which handles basic text emission.

> classes ? If so, should we emit directly NASM code or should we keep an
> additional intermediate layer and emit X# code ? One way or another,
> this means simple assembler instructions text such as the "MOV AX, 0x00"
> one you exemplified would be directly emitted by IL op classes from the
> Cosmos.IL2CPU assembly.

These IL Op classes will be converted to X# instead.

> You're not talking about plugs. Some of them rely on Cosmos.Assembler
> for NASM source code emission. Do you intend them to be migrated the
> same way than IL classes above ?

Yes they need migrated as well to X#.

> The XShap.Nasm assembly and its Assembler class is what I understand to
> be the "new simplistic assembler class". It looks like you expect it to
> be extended with new methods as part of step 1. What kind of features do
> you expect to be added during this step ?

Whatever is needed, but this class should remain rather simple and
mostly handle streaming etc.

> From my point of view step 1 and 2 looks very tightly coupled with each
> other and having them split into two steps looks a little bit
> artificial, unless additions to XSharp.Nasm.Assembler class is much more
> work than I can foresee.

XSharp.Nasm is pretty simple but isolates X# from knowing where the text
goes etc.