Ironically, this is actually why I am considering buying a windows machine for the first time in a very long time. NVDA gpus take linux back to the old days that you have to deal with all these possible issues. If you build a machine , you pretty much are stuck with OpenCL and not CUDA unless are fine with random system and compatibility issues. Then if you price things out, we are really in a golden age of gaming laptops. A gaming laptop with a NVDA card though is just not going to be worth even bothering with linux. You have to get a Lenvo without a NVDA card if you want a linux laptop. I am going to end up getting a Windows gaming laptop to run local models and use Windows the same way I use it at work. Basically, assume everything is being monitored. I think right now this is the ideal setup. Windows gaming laptop for a GPU and Linux for basically everything else. As much as I hate Windows, there are a few pieces of software I miss dearly. Especially when it comes to audio software.
This is good feedback, much appreciated. My daily driver laptop has an NVDA chip in it, maybe I should just play with the local models until I bump up against the hardware limitations.
This is true. My current desktop has an Nvidia GPU and there's a lot of issues with sleep. But I know this stems from Nvidia (deliberately?) making nasty drivers for Linux and not the Linux kernel devs. I fully agree with Torvalds and his middle finger gesture to Nvidia. Any models I use are CPU based for now so I'd just get an AMD GPU. I have no need for the Nvidia nonsense.
You guys didn't hear about this lol? They only just announced this week that they were going to remove it.... but one has to ask.... what friggin' world do these people in Redmond live in?!! I mean it probably took at least a year or so to develop this.... was there no one there that had enough common sense to say "Um... hey guys... I don't think this is gonna go over real well." New Windows AI feature records everything you’ve done on your PC Recall uses AI features "to take images of your active screen every few seconds." Benj Edwards - May 20, 2024 9:43 pm UTC Enlarge / A screenshot of Microsoft's new "Recall" feature in action. Microsoft 448 At a Build conference event on Monday, Microsoft revealed a new AI-powered feature called "Recall" for Copilot+ PCs that will allow Windows 11 users to search and retrieve their past activities on their PC. To make it work, Recall records everything users do on their PC, including activities in apps, communications in live meetings, and websites visited for research. Despite encryption and local storage, the new feature raises privacy concerns for certain Windows users. Further Reading Microsoft’s “Copilot+” AI PC requirements are embarrassing for Intel and AMD "Recall uses Copilot+ PC advanced processing capabilities to take images of your active screen every few seconds," Microsoft says on its website. "The snapshots are encrypted and saved on your PC’s hard drive. You can use Recall to locate the content you have viewed on your PC using search or on a timeline bar that allows you to scroll through your snapshots." By performing a Recall action, users can access a snapshot from a specific time period, providing context for the event or moment they are searching for. It also allows users to search through teleconference meetings they've participated in and videos watched using an AI-powered feature that transcribes and translates speech. At first glance, the Recall feature seems like it may set the stage for potential gross violations of user privacy. Despite reassurances from Microsoft, that impression persists for second and third glances as well. For example, someone with access to your Windows account could potentially use Recall to see everything you've been doing recently on your PC, which might extend beyond the embarrassing implications of pornography viewing and actually threaten the lives of journalists or perceived enemies of the state. Despite the privacy concerns, Microsoft says that the Recall index remains local and private on-device, encrypted in a way that is linked to a particular user's account. "Recall screenshots are only linked to a specific user profile and Recall does not share them with other users, make them available for Microsoft to view, or use them for targeting advertisements. Screenshots are only available to the person whose profile was used to sign in to the device," Microsoft says. Users can pause, stop, or delete captured content and can exclude specific apps or websites. Recall won't take snapshots of InPrivate web browsing sessions in Microsoft Edge or DRM-protected content. However, Recall won't actively hide sensitive information like passwords and financial account numbers that appear on-screen. Further Reading New Mac app wants to record everything you do—so you can “rewind” it later [Updated] Microsoft previously explored a somewhat similar functionality with the Timeline feature in Windows 10, which the company discontinued in 2021, but it didn't take continuous snapshots. Recall also shares some obvious similarities to Rewind, a third-party app for Mac we covered in 2022 that logs user activities for later playback. As you might imagine, all this snapshot recording comes at a hardware penalty. To use Recall, users will need to purchase one of the new "Copilot Plus PCs" powered by Qualcomm's Snapdragon X Elite chips, which include the necessary neural processing unit (NPU). There are also minimum storage requirements for running Recall, with a minimum of 256GB of hard drive space and 50GB of available space. The default allocation for Recall on a 256GB device is 25GB, which can store approximately three months of snapshots. Users can adjust the allocation in their PC settings, with old snapshots being deleted once the allocated storage is full. As far as availability goes, Microsoft says that Recall is still undergoing testing. "Recall is currently in preview status," Microsoft says on its website. "During this phase, we will collect customer feedback, develop more controls for enterprise customers to manage and govern Recall data, and improve the overall experience for users."
Anyone on this site who changed their nick because of AI should be instablock, even though I am against blocking I mean, WTF!
CUDA works fine under RHEL on H100 (for example), though you have to use their proprietary drivers. It’s the consumer grade hardware that has problems.
This is all the problem I am have been trying to figure out for the past month. There is just so many options. I feel like I have put off not having a GPU for years and sticking with cpu models like d08. My current desktop though is really long in the tooth and ready for an upgrade but I think I am going to end up with laptops for mobility. The macbook air is another variable in all this. As cool as a Thinkpad would be with linux, it seems like it would make more sense to just get a macbook at that point. So long term, I think I am going to end up with a Windows MSI gaming laptop for the Nvidia card and a macbook for personal use with no linux. This is even though if all things were equal I would just use linux on everything. The other option is to just rent a cloud gpu on something like runpod or vast.ai. I have runpod open and you can rent a RTX 4090 with 24 gigs of ram for 49 cents an hour spot price right now. That is per GPU and you could rent 8 of them for $3.92 an hour. The problem is I just hate not having things on my local machine. I also think if I have my own GPU, I will be highly motivated to run things on it and learn more.