Replace "." With "" Different Effect When Used On Text And Numbers

I make downloads from SAP and they come in the following format:
#.###,##

This is the number format we usually use in Norway.

The problem is that Excel don't understand that for example 1.234,99 is a number because of the ".". They way to fix this manually is to use Replace "." with "". This translates into the following VBA

Selection.Replace What:=".", Replacement:="", LookAt:=xlPart, _
SearchOrder:=xlByRows, MatchCase:=False, SearchFormat:=False, _
ReplaceFormat:=False

Using this macro will make the following changes:
1.234,99 -> 1234,99
4,2 -> 42

The first is exactly what I wanted (text -> number), but an unwanted side effect is that Excel believes that "." is the decimal separator when faced with a number.

An obvious way to solve this would be to convert everything into text first, but the only way I could think of to do this is as follows:

1) Adding a character before or after every number in a new column (number -> text)
2) Replace "." in the new column
3) Delete character (text -> number)
4) Copy back to the original column